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WITH MOBILITY 
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, INRIA and Universite Paris 7 

, Cu ■ A stochastic model for a mobile network is studied. Users en- 

ter the network, and then perform independent Markovian routes 
between nodes where they receive service according to the Processor- 
Sharing policy. Once their service requirement is satisfied, they leave 
the system. The stability region is identified via a fiuid limit ap- 
proach, and strongly relies on a "spatial homogenization" property: 
at the fiuid level, customers are instantaneously distributed across the 
network according to the stationary distribution of their Markovian 

iS^ • dynamics and stay distributed as such as long as the network is not 

jrt I empty. In the unstable regime, spatial homogenization almost surely 

holds asymptotically as time goes to infinity (on the normal scale), 
telling how the system fills up. One of the technical achievements of 
the paper is the construction of a family of martingales associated to 

^S] ' the multidimensional process of interest, which makes it possible to 

^ ' get crucial estimates for certain exit times. 

in 

o 

CN ■ 1. Introduction. Recent wireless technologies have triggered interest in 

a new class of stochastic networks, called mobile networks in the technical 
C^ ' literature [3, 11]. In contrast with Jackson networks where users move upon 

^rz • completion of service at some node, in these mobile networks, transitions of 

f~^ I customers within the network occur independently of the service received. 

Moreover, at any given time, each node capacity is divided between the 
users present, whose service rate thus depends on the capacity and on the 
r> I state of occupancy of the node. Once his initial service requirement has been 

C^ • fulfilled, a customer definitively leaves the network. In [3], complex capacity 

sharing policies are considered, but in the simplest setting, which will be of 
interest to us, nodes implement the Processor-Sharing discipline by dividing 
their capacity equally between all the users present. Previous works [3, 11] 
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have mainly focused on determining the stabihty region of such networks, 
and it has been commonly observed that the users' mobility represents an 
opportunity for the network to increase this region. Indeed, because of their 
mobility, users offer a diversity of channel conditions to the base stations 
(in charge of allocating the resources of the nodes), thus allowing them to 
select the users in the most favorable state. Such a scheduling strategy is 
sometimes referred to as an opportunistic scheduling strategy (see [2] and 
the references therein for more details). 

In the present paper, we investigate from a mathematical standpoint a 
basic Markovian model for a mobile network derived from [3] . In this simple 
setting, customers arrive in the network according to a Poisson process with 
intensity A, and move independently within the network, according to some 
Markovian dynamics with a common rate matrix Q. Service requirements 
are exponentially distributed with mean 1, and customers are served at each 
node they visit according to the Processor-Sharing discipline, until their 
demand has been satisfied. The total capacity of the network, defined as 
the sum of all the individual capacities of the nodes, is denoted by /i. It 
corresponds to the instantaneous output rate of the network when no node 
is empty, that is, when there is at least one customer at each node. 

It is of particular interest to note that, even if Q is reversible, because of 
the arrival and departure processes, the system is not reversible. This con- 
trasts with earlier works in which particle systems with similar dynamics 
have been investigated under reversibility assumptions. In [6], the authors 
look at a closed system (i.e., with parameters A = /i = 0) where transition 
rates are chosen such as to yield a reversible dynamics. In this case, the sta- 
tionary distribution of the system has a product form, and the authors are 
interested in showing that the convergence to equilibrium is exponentially 
fast. Their approach essentially relies on logarithmic Sobolev type inequali- 
ties. 

In our case, however, a different set of questions is addressed, involving 
different tools. Since the system under consideration is open, it may be 
unstable, so that a natural issue is to determine the stability region. We 
prove, as was conjectured in [3], that the intuitive, simple condition A < // 
is indeed the stability condition (the critical case X = fj, is not considered) . 
In contrast with Jackson networks for which the stability condition is local, 
in the sense that each node has to satisfy some constraint, here only the 
global quantities A and fi matter. This shows that mobility allows to make 
the most of the potential service capacity of the network, corroborating the 
results previously mentioned. Note that X < fi being a necessary condition is 
obvious, since /i is the maximal output rate. But surprisingly, proving that it 
is sufficient requires very technical tools, including the use of fluid limits and 
martingale techniques. In particular, the long and tedious Appendix is solely 
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devoted to the construction of a martingale which provides key estimates 
for showing that \ < fi corresponds to a stable system. 

This martingale is a multidimensional (therefore complicated) general- 
ization of the martingale built in [10] for the M/M/oo queue, and this is 
not completely surprising, since as will be seen, the model inherits salient 
properties of the M/M/oo queue. Besides, the construction of a martingale 
associated to a multidimensional process represents one of the technical 
achievements of this paper: such examples are indeed pretty scarce in the 
literature. Similar to [10], the approach relies on building a family of space- 
time harmonic functions indexed by some parameter c € M", and then on 
integrating over c in such a way as to preserve the harmonic property. 

Through studying both the stability region and the unstable regime, a 
detailed description of the behavior of the system is given, resulting in two 
versions (stable and unstable) of the following rough property: when many 
users are present in the network, they get approximately distributed among 
the nodes according to the unique invariant distribution vr associated with 
Q, the latter being assumed irreducible. It must be emphasized that yet, 
contrary to [3], customers' movements are not assumed stationary. 

As a first argument for this spatial homogenization, the law of large num- 
bers suggests that, when the total number of users initially present in the 
network is large, the proportions of users at the different nodes should be 
close to vr after some time, related to the convergence to vr of the Markov 
process associated to Q. The more delicate question that next arises, of 
how long these proportions stay close to vr, constitutes the main challenging 
issue of the paper, that requires martingale techniques for estimating the 
deviation time from vr. 

The short-term reach of vr is understandable from an analogy with the 
M/M/oo queue; indeed, independence of the customers' trajectories yields 
that, similar to the M/M/oo queue, the output rate from any node due 
to inner transitions is directly proportional, through Q, to the number of 
customers at this node. When the network is overloaded, the relative oc- 
cupancies of the nodes should then, after a while, be close to the internal 
traffic balance ratios given by vr. 

A more explicit analogy with another classical queueing model is provided 
by the following simple but crucial observation: as long as no node is empty, 
the total number of customers simply evolves as an M/M/1 queue with 
input rate A and output rate fi. This is the case, in particular, when the 
distribution of customers is close to vr. This interplay between, on one hand, 
the proportions of customers at the different nodes, and, on the other hand, 
their total number, will underly the analysis throughout the paper. 

While the short-term behavior, which results in the spreading of customers 
according to vr, is dominated by the M/M/oo dynamics, the long-term be- 
havior is essentially driven by the M/M/1 dynamics of the total number 
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of customers. This naturally suggests that two different scalings have to be 
considered: one, corresponding to the M/M/oo dynamics where only space 
is scaled and not time; and a second one where both space and time are 
scaled corresponding to the fluid scaling of the M/M/1 queue. Note that 
the natural scaling for the M/M/oo queue is the so-called Kelly scaling, in 
which space and input rate are scaled. Here, since the input rate at each 
node due to inner transitions is a linear function of the numbers of cus- 
tomers at the different nodes, there is no need to scale the external input 
rate A. Inner movements dominate the dynamics and the space scaled pro- 
cess converges, analogously to the M/M/oo queue under Kelly's scaling, to 
some deterministic trajectory, with limit point at infinity here given by vr. 

The coexistence of these two different scalings makes the use of fluid 
limits both original and challenging. Fluid limits are a standard tool in the 
analysis of complicated stochastic networks. Rybko and Stolyar [13] is one 
of the first papers that uses this technique together with Dai [8]. Dupuis 
and Williams [9] present similar ideas in the context of diffusions. In a series 
of papers, Bramson [4, 5] describes the precise evolution of fluid limits for 
various queueing networks. See also the books by Chen and Yao [7] and 
Robert [12]. In the context of networks, fluid limits have been used mainly 
for Markov processes which behave locally as random walks. For this reason, 
results related to fluid limits are sometimes presented as functional laws of 
large numbers. Because of the mixture of two different dynamics, given by 
the M/M/1 and M/M/oo models, our framework is somewhat different. A 
second important difference with the existing literature concerns tightness 
results which are usually easy to obtain, mainly because transition rates are 
generally bounded; this not the case here. 

The long-term analysis is twofold. Deriving fluid limits requires a control 
on the process over time periods of the same order as the initial number of 
customers (since the fluid scaling parameter is the same for time and space) . 
In the stable case this is obtained by showing that the deviation time from 
vr is essentially larger than the time for the underlying M/M/1 queue to 
empty. The unstable case exhibits a more striking behavior: the deviation 
time from vr is not only large compared to the initial number of customers, 
but is even infinite with high probability. This amounts to a control of the 
whole trajectory; the distribution of users among nodes stays trapped in 
any neighborhood of vr with high probability as the initial state is large. 
This result is related to a strong convergence result stating that, for any 
fixed (nonscaled) initial state, the system almost surely diverges along the 
direction of tt. Note that a similar phenomenon has been exhibited in [1], in 
the context of branching Markov chains, that is, Galton- Watson branching 
processes where individuals located at some countable set of sites move at 
their birth time. 
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These various remarks and outline of results lead to the following organi- 
zation for the paper. Section 2 gives a precise description of the stochastic 
model and introduces the notation that will hold throughout the paper. 
We have already mentioned the construction of a martingale which gives 
important estimates through optional stopping techniques. Section 3 intro- 
duces this martingale, and provides the main estimate that will be used. 
Due to its technicality, the construction of the martingale is postponed to 
the Appendix. 

Section 4 establishes a decomposition of the process as, mainly, the dif- 
ference between two processes of the same type but with no departures. 
For such a process (with null service capacity), a representation involving 
labelled particles is given. Both representations will help derive the almost 
sure convergence result of Section 6. 

The three last sections are devoted to analyzing the behavior of the sys- 
tem. Section 5 deals with the short-term behavior, thus studying the only 
space renormalized process. Section 6 studies the supercritical case A > /i, 
establishing among other results the almost sure convergence of the propor- 
tions to the equilibrium distribution vr as t — t- oo. Finally, Section 7 proves 
the stability of the system in the subcritical case A < //. 

2. Framework and notation. This section gives a precise description of 
the model under consideration and introduces the main notation. The net- 
work is described by a Markov process X = {X{t),t > 0) characterized by 
its infinitesimal generator, given by (1) below. 

Section 6 will make use, in the particular case of null service capacity, of 
a more explicit representation of X involving a sequence of Markov jump 
processes that represent the trajectories of the successive customers entering 
the network. The general description of the system through its Markovian 
dynamics provided in the present section is, however, sufficient for most 
results of the paper, especially for building a family of martingales and for 
determining the stability condition. 

The network consists of n nodes between which customers perform in- 
dependent (continuous-time) Markovian routes during their service. In this 
setting, transitions of customers from one node to another are driven by 
some rate matrix Q = {qij, 1 <i,j <n) and are thus not triggered by service 
completion. 

New customers arrive at node i = 1, . . . ,n according to a Poisson pro- 
cess with intensity Aj > 0, and then move independently according to the 
Markovian dynamics defined by Q. The arrival processes at the different 
nodes are independent, so that the global arrival process is Poisson with in- 
tensity A = ^" Aj . The case A = corresponds to a system with only initial 
customers, and no new arrivals. 



6 F. SIMATOS AND D. TIBI 

Upon arrival, or at time i = for those initially present, customers gener- 
ate a service requirement which is exponentially distributed with mean 1. All 
service requirements, arrival processes and Markovian routes are assumed 
to be mutually independent. 

Node i, 1 < i <n, has service capacity /Uj > 0, which is divided at any 
time between the customers present, according to the Processor-Sharing 
discipline: if A'^ is the number of customers present at node i, then each 
of these A^ customers is served at rate fJ-i/N. The service rate of a given 
customer thus evolves in time, depending on his current position and on 
its occupancy level. Once a customer has received a service that meets his 
initial requirement, he leaves the network. 

The total service capacity of the network is defined as /U = Xli /"«• ^O" 
tice that, due to the exponential nature of the services, the mechanism of 
departure from one node by completion of service does not distinguish the 
present Processor-Sharing discipline from the FIFO discipline: the instanta- 
neous output rate from the system at node i is //j, provided that node i is 
not vacant. The total output rate is then /i when no node is empty. 

The process of interest is A = {X{t),t > 0) defined by 

A(t) = (Ai(t),...,A„(t)), t>0, 

where Aj(t), for i = 1, . . . ,n, is the number of customers present at node 
i at time t. The Markovian nature of the movements together with the 
exponential assumption for the service distribution imply that A is a Markov 
process in N" with infinitesimal generator fi given, for any function / : N" — )• 

M and any x = (xi, . . . , Xn) G N", by 

n 

i=l 
n 

(1) +Y,^..>o}^^^{f{^-ei)-f{x)) 

i=l 

+ ^ qijXi{f{x + ej-ei)- f{x)), 

l<iy^j<n 

where Cj G N" has all coordinates equal to 0, except for the ith one, equal 
to 1. 

The Introduction has highlighted that this system is a mixture of two 
classical models in queueing theory, the M/M/1 and the M/M/oo queues. 
This is readable in the expression of the generator given in (1) where the 
two first sums are reminiscent of the M/M/1 queue and the last one of the 
M/M/oo queue. 
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The rate matrix Q is assumed to be irreducible, admitting vr = (vtj,! < 
i <n) as its unique stationary distribution characterized by the relation 

ttQ = 0. 

For technical reasons related to the construction of the martingale intro- 
duced in Section 3 (see in the Appendix) , we require the additional assump- 
tion that Q is diagonalizable. This assumption is satisfied if Q is reversible 
with respect to vr, but it is in general a much less restrictive constraint. 

For any t > 0, the random vector X{t) will often be described in terms of 
the total number of customers L(t) and the proportions of customers at the 
different nodes x(^) = (Xi(^)i 1 <i <n). More formally, define 

" X (t) 

L{t) = Y,Xj{t) = \Xit)\ and xi{t) = ^, l<i<n,t>0, 

with the convention that xi^) = ^i when L{t) = 0. Here, and more generally 
for any x = {xi, . . . ,x„) G M", |x| denotes the i^ norm in M" : |x| = ^" |xj|. 
The vector x(t) can be identified with a probability measure on {1, . . . , n}: 
namely, the empirical distribution of the positions of the L{t) customers 
present in the network at time t. Denote by 



p=JpG[0,+oor:^/5, = l 



O 

the state space of x(0- The interior set of V is V = {p G ]0, +cx)[" : ^" pi = 

!}• 

As emphasized earlier, the deviation of xi^) from tt will be of particu- 
lar interest in the forthcoming analysis. It will be measured, depending on 
circumstances, by the i°^ distance ||x(*) — '''"II) 

ll^ll = max IxjI, X = (xi, . . . ,x„) G M", 

l<i<n 

or by the relative entropy iif(x(i),7r) where H{-,Tr) is defined on the set V 
of probability measures on { 1 , . . . , n} by 

n 

H{p,7r) = y^p,log^e[0,+^[, per. 

For t > 0, the quantity f/'(x(t),vr) will also be more simply denoted H{t). 
The process {H{t),t > 0) will spontaneously appear in the expression of the 
key martingale Ja introduced in the next section. 

The different deviation times of x(t) from tt, or conversely, the time needed 
for x(t) to reach a given neighborhood of vr, will be of particular interest. 
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For e > 0, T^ (resp. T^) denotes the first time when the i°° distance between 
x(t) and vr is smaller (resp. larger) than e, 

Te = inf{t>0:||x(i)-vr|| <e} and T" = mf{t>0:\\x{t) - tt\\ > e}. 

Most results will be written in terms of these two stopping times, but it will 
be sometimes more convenient to work with the deviation time Tfj from vr 
in terms of the relative entropy, 

Tfj = mi{t>0:H{t)>e}. 

All results on deviation times of x(^) from vr defined in terms of the £°° 
distance \\xit) — tt\\ can be translated into analogous estimates in terms of 
the relative entropy H{t) thanks to the following classical result: 

Lemma 2.1. There exist two tt -depending positive constants Ci and C2 
such that, for all p £V, 

Cl\\p-TTf<H{p,7T)<C2\\p-7Tf. 

In particular, for any e > 0, T^^^ <T^ < "^h^^ ■ 

Another stopping time will play a central role, namely, the first time, 
denoted by To, when the system has an empty node. Formally, 

% = inf{t > : 3i G {1, . . . ,n},Xi(t) = 0}. 

Indeed, the martingale property for the family of integrals presented in Sec- 
tion 3 will hold only up to time To, that is, as long as the output rate at 
each node i is exactly equal to /Xj. In the same way, it will be easily shown 
that, for t < To, L{t) behaves exactly like the M/M/1 queue with input rate 
A and output rate p. 

A last useful remark concerning these stopping times is that, when To is fi- 
nite, ||x('7o) ~ '''"II ^ min7rj(> 0). Together with Lemma 2.1, this immediately 
gives the following result: 

Lemma 2.2. There exists eo > such that T^ V Tfj < To holds for any 
e<£o. 

3. Martingale. The results of this section are twofold: Theorem 3.1 gives 
the (almost) explicit expression of a local martingale Jq(- A To), indexed by 
some positive parameter a, and Proposition 3.2 derives the main estimate 
on deviation times Tfj of x(^) from tt, that will be used in Sections 6 and 
7. Concerning the construction of Ja, the present section only aims at giv- 
ing the main lines. The (numerous) technical details are postponed to the 
Appendix. 
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The approach for constructing the martingale J^ is similar to the ap- 
proach used in [10] for the M/M /oo queue. The idea is to first exhibit a 
family of space-time harmonic functions {hv{t,x),v E M"') for the genera- 
tor il given by (1), and then to integrate hy(t,x)f{v) with respect to v for 
some suitable function /, on some well-chosen, time-dependent domain. The 
last step is then to make a change of variables so that the new harmonic 
function is split into two factors, respectively, depending on time and space. 
The resulting local martingale is then adapted for an optional stopping use, 
leading to hitting-times estimations. 

Some notation are required at this point. Denote by {Pt,t £ M) the Q- 
generated Markov semi-group of linear operators in M."' : Pt = e**^, extended 
to all real indices t into a group. For v E M" and t E M, define 

cj){v, t) = {(l)i{v, t),l<i<n) = P.tv. 

Theorem 3.1 below requires the technical assumption that Q is diagonal- 
izable. Let 9 be the trace of —Q, so that 9 > 0, and let S C M""'^ be the 

o 

projection of V G M" on the n — 1 first coordinates, that is, 

r _ n-i >! 

5 = < n = (ui, . . . ,tin-i) E M" : Vz = 1, . . . ,n — l,Mj > and /^^j ^ ^ f • 

o 

For any u€z S, denote by u E "P the nth-dimensional vector which completes 
u into a probability distribution, that is, Ui = Ui for any 1 <i <n — 1 and 

Un = l-2^l Ui. 

The following proposition describes a family of space-time harmonic func- 
tions. 

Proposition 3.1. Let v E M" be fixed, and let (p{v,-) be any primitive 
of 



|:(-rS -•«-)) 



on any open subset V o/ {t > 0: 1 + (/)i(f ,t) 7^ for i = 1, . . . ,n}. The func- 
tion 

n 

K{t,x) = e'^('''*) '[[{l + 4>i{v,t)f\ t E y,x E N", 

is space-time harmonic with respect to Q in the domain V x N*". 

Proof. It must be shown that dhy{t,x)/dt + Q{hv{t,-)){x) = on the 
above domain. For x E N*" and t £V, hv{t,x) ^ 0, and one easily computes 

1 dhy dip Y^ d(f)i{v,t)/dt 



h,{t,x) 9t"^*'^^" St^'^'^^ + ^^^l + ^^Kt) 
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and 

(pi{v,t) 



4>j{v,t) -(t)i{v,t) 



hv{t,x) ' ~[ ' ~[ l + (t)i{v,t) 



+ 2_^ XiQij 



The last term in the right-hand side is equal to X^ILi i+s\v t) (Q'^(^iO)»- 
By definition cj) satisfies d(j){v,t)/dt = —Q(l){v,t), and the result follows. 

D 

Remark 3.1. The product form of these space-time harmonic functions 
is quite similar to that of the harmonic functions introduced in [10] for the 
M/M/oo queue. 

In addition, it is easily checked that, choosing f = (n — 1, . . . ,u — 1) for 
some u^O, so that v is some eigenvector of P^, i G R, associated with eigen- 
value 1, yields K{t,X{t)) =u^(t)^[x{i-u)+f,{i-i/u)]t^ which is the martingale 
associated with an M/M/1 queue L with arrival rate A and service rate // 
(see, for example, [12]). 

Starting from hy{t,x), two steps lead to J^: (i) integration of hy{t,x) over 
V against some function f(v) on a suitable time-dependent domain T>(t); 
(ii) change of variables. These two steps are detailed and justified in the 
Appendix, yielding the following family of local martingales: 

Theorem 3.1. There exist two positive, continuous, bounded functions 

o 

F and G onV such that for any a > 0, u i— )• F(n)"~ is integrable on S and 
(jQ,(t A To), i > 0) is a nonnegative local martingale where Ja{t) is defined 
for a > and t>0 by 



Ut) = e-^'' j nf^J G{u)F{ur-Uu, 



or equivalently, 



(2) Ut) = e-°^* / e^W(^W-^(^W'"))G(n)F(ti)"-i du. 
Moreover, F satisfies 

(3) sup (a" / F(ii)"~i du] < +00. 

0<a<lV Js J 
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The advantage of Ja{t) [as compared to hy{t,X{t))], is that the depen- 
dence in time is there split into two factors: e~"^* is a direct function of 
time, and the integral is a function of the state of the system at time t, X{t) 
or equivalently {L{t),xit))- 

The next proposition gives the fundamental estimate obtained through 
optional stopping and is used several times throughout the paper. 

Proposition 3.2. For any 5 such that < 5 < Eq, where £o is given by 
Lemma 2.2, there exists some constant Cg such that 

holds for any initial state x G N" and any e ^]5^£q[, ^ > and a G ]0, 1] . 

Proposition 3.2 is derived from the two following lemmas by choosing 
T = Tfj (so that, by Lemma 2.2, T ATo = T when e KEq). Note that only 
Lemma 3.1 uses the fact that J^ is a local martingale, whereas Lemma 3.2 
stems directly from the expression of Ja provided by (2). 

Lemma 3.1. There exists some constant C3 > such that, for any a G 
]0, 1], any initial state x £ N" and any stopping time T, the following in- 
equality holds: 

Proof. Fix a G ]0, 1] and x G N". Since Ja{- A To) is a nonnegative local 
martingale, it is a supermartingale, and so is {Ja{tAT ATo),t > 0) by Doob's 
optional stopping theorem. In particular, for any t > 0, 

iE.[Ja(o)]>E,[j,(tArAro)] 

and Fatou's lemma gives 

Ex[Ja(0)] >liminfE^[J„(tArAro)] 

t— ^+00 



>E, 



liminf^(iATAro) = E^[J„(r A 7^)] 

. t— >+oo J 



[here Ja(T A To) makes sense a.s. when T A To = +00 since any nonnegative 
supermartingale almost surely converges to some variable at infinity]. 
Prom the definition of Ja given by (2), using e^^ < 1 for y > 0, one gets 

^x[Ja{0)] < sup(G)el^'l^(^/l^l'") / F{u)"-^ du < Cael^l-^^^/l^l'^^a"", 

where C3 = sup o (G) sup„<]^(a"' /^ F{'u)°'~^ du) is finite by (3), which proves 
the lemma. D 
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Lemma 3.2. For any positive 5, there exists some positive constant Bs 
such that the following implication holds for any a G ]0, 1], i> 0, e > 5 and 
t>0: 



L{t)>i and H{t)>e =^ Jait)>Bs-e 



-aet+(e-5)e 



Proof. Fix e > 5, a g]0, 1], £ > and t > 0. A lower bound on the 
integral part of (2) is obtained when L{t) > i and H{t) > e. For v & V, 
define the set Ss{v) C 5 by 

Ss{v) = {u e S : H{v,u) < 6}. 

If H{t) > e and L(i) > £, then 

^L{t)iH{t)-H{x{t\u)) G{u)F{ur-^ du > /3e^(^-^) / G{u) du, 

S JSs{x(t)) 

where /3 = niin{(sup o F)~^, 1}. Indeed, a being smaller than 1, /3 is a lower 
bound for F(n)"~^ on S. Consider now the function ^s'-'P ^ I^^ defined by 

^s{v)= I G{u)du. 

Jss(v) 

Since G is bounded, $5 is easily shown to be continuous (using, e.g., Lebesgue's 
theorem). Moreover, ^^{v) > for any v gV [because G > and the inte- 
rior of Ss{v) is not empty], and since V is compact, infp <&5 > 0. Setting 
Eg = /3 inf-p $5 achieves the proof. D 

4. Two key representations. The Markov process {X{t),t > 0) with in- 
finitesimal generator Q defined by (1) can be seen as a particle system involv- 
ing three types of transitions: births, deaths and migrations of particles from 
one site to another. The main purpose of this section is to show that X can 
be decomposed into the difference of two pure birth and migration processes, 
up to some refiection term (Theorem 4.1). A simpler result (Proposition 4.1) 
tells that, as long as X does not hit the axis, the process L of the total num- 
ber of particles just behaves as a random walk (or equivalently as an M/M/1 
queue). Finally, a representation of process X involving labelled particles is 
given in the case of null death rates. 

Theorem 4.1, together with the latter representation, will be crucial for 
describing the unstable regime in Section 6, while Proposition 4.1 will be 
repeatedly used in the study of both the super and subcritical regimes. 

The idea for decomposing X is the following: when /i = 0, the system con- 
sists of immortal particles generated at rate A and performing independent 
Markov trajectories. Introducing a death procedure, that is, some positive /i, 
amounts to eliminating particles (at rate fii at site i) if possible. Up to some 



{x,y,z) 
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correction, due to the fact that no death can actually occur at an empty site, 
this is equivalent to subtracting some analogous process with birth rates fii 
(1 < i < n), zero death rates and migration rate matrix Q. 

This can be formalized by introducing an enlarged Markov process in- 
volving three types of particles. Define {X, Y, Z) as a Markov process in N^" 
with generator T characterized by the following transitions and rates: for 
any (x, y, z) G N^" and i, j G {1, . . . , n} such that i / j, 

(x + ei,|/,z), at rate Aj, 

{x-ei,y + ei,z), ^il{^^>i}, 

{x,y,z + ei), ^jl{^^=o}> 

[X Ci -\- Cj ,y, Z), QijXi , 

{x,y-ei + ej,z), qijyi, 

[x,y,z e j -\- Cj ), Qij Zi . 

(X keeps track of the "real" particles, Y of the killed ones and Z of virtual 
particles generated at some site when no particle has been found to be 
killed.) 

It is clear from these transitions and rates that, indexing generator fi by 
its birth and death rate vectors: A = (Aj, 1 < i < n) and /i = (^j, 1 < i <n) 
(A, /i G [0, +oo["') and denoting by the null vector in M": 

(i) X is a Markov process in W" with generator il,x^^; 

(ii) X + Y is also Markov in N", with generator ilA,0J 
(iii) y + Z is Markov in N" with generator ri^^oj 
(iv) |X + y| — (1^(0) + y(O)l) is some Poisson process with intensity A; 

(v) \Y + Z\ — (|1"(0) + .^(0)1) is some Poisson process with intensity /n; 
(vi) these two Poisson processes are independent. 

Now from (i), any process X with generator fi can be considered as the 
first component of some Markov process with generator T and with initial 
state (X(0),0,0). 

The two next results are easily derived from this construction and from 
remarks (i) to (vi). In order to state the main theorem, it is convenient to 
index the process X both by its initial state and by its birth and death pa- 
rameters, writing X^ for the process X with initial state x G N", migration 

rate matrix Q and birth (resp. death) parameters A = (Aj,l <i<n) [resp. 
/f = (/"i,l <i <n)]. 

Theorem 4.1. For any x G N" and X,fx G [0, +oo[", there exist versions 
of Xf , Xf^ Q and X- q such that 

XX \^x v^ I v 

where Z is an W^ -valued process such that \Z\ is nondecreasing, initially 
zero, and increases only at times when some Xi{t) is zero. 
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Proof. Write X = X + Y - {Y + Z) + Z, where {X{0),Y{0),Z{0)) = 
(x, 0, 0) and (X, Y, Z) is Markov with generator F, so that X is some version 
of X^ , and by (ii) and (iii) , X + Y is some version of X^ q and Y + Z some 

version of -'^"q- 

The theorem is proved, since \Z\ has the stated properties as can be seen 
on r. □ 

Remark 4.1. The process Z in Theorem 4.1 appears as a reflection 
term; it guarantees that X stays nonnegative, compensating by adding some 
virtual particle for a jump of X-q that would get some Xi to the value —1. 

However, contrary to usual multidimensional Skorokhod reflection terms, 
here, due to the movements of particles, components Zj's are not necessarily 
nondecreasing in time; only their sum is. Also, Zi can increase at times when 
Xi is not zero. 

Theorem 4.1 and its proof, together with properties (iv), (v) and (vi), 
give the following proposition which constitutes one of the key ingredients 
for deriving the fluid limits in Sections 6 and 7. 

Proposition 4.1. For all t<To, the following equality holds: 

L{t) = L{0)+Mx{t)-M^{t), 

where Mx and N^ are independent Poisson processes with respective inten- 
sities A and /i. Moreover, L{t) > L{Q) +N\{t) — J\ffj,{t) holds for any t > 0. 

We conclude this section with a representation of process X^ q that will 
notably be used in Section 6, in conjunction with Theorem 4.1, for analyz- 
ing the unstable regime. X^q is here obtained as a function of a Poisson 
process with intensity A and a sequence of Markov processes with infinitesi- 
mal generator Q (representing the trajectories of the successively generated 
particles) . 

More precisely, X^^ admits the following representation: 

(4) Xl^{t)=(Y.M(.it-a,)=^,a,<t}^<i<n), i > 0, 

where: 

• (7fc = for 1 < A; < |x| ; 

• A/a = (o"fc, /c > |x| + 1) is a Poisson process with parameter A; 

• S,k, k>l, are Markov jump processes in {1, . . . ,n} with generator Q and 
initial distribution: 

- Etli>^i/>^)^^^0^'k>\x\+l■ 



SPATIAL HOMOGENIZATION IN A MOBILE NETWORK 15 

- 5i for Xi arbitrarily chosen indices A;G{l,...,|rc|} (l<i<n); 
• A/a and the S,k, k>l, are mutually independent. 

iCkj 1 ^ ^ ^ |a;|) hold for the trajectories of the initial particles and {^k^k > 
|x| + 1) for those of the successive newborn particles. M\ holds for the global 
birth process. For k>l, particle k is in the system from time a^ {a^ = for 
k< \x\). 

Similarly as in the previous construction, a formal proof of (4) can be 
provided by constructing X^ q as function of a more complete process (that 
also contains J\f\ and the ikS, A; > 1), characterized through its infinitesimal 
generator and describing the list of current positions of particles present in 
the system ordered according to their birth rank. 

5. The space renormalized process. The stability property of the system 
for X < fj, will be derived in Section 7 from a fluid scaling analysis, that is, 
from the study of the space-time renormalized process 

\x\ 

as |x| goes to infinity where X^ is the Markov process X initiated at x. It 
will be underlain by the M/M/1 behavior of the total occupancy process L 
(only valid as long as no Xi is zero, hence the intricacies of the analysis). 

The particular behavior of X at i = 0"*" will result from the short-term 
behavior of the only space renormalized process X, defined as the the family 
of processes 

X^t) = ^^, t > 0, for X G N" \ {0}. 

[The simpler notation X{t), where X{t) = X(t)/\X{0)\, will also be used in 
situations where |-Y(0)| is clearly nonzero.] 

As highlighted in the Introduction, this scaling is natural and analogous 
to the Kelly scaling for the M/M/oo queue. This analogy appears in Propo- 
sition 5.3 below, that states convergence of X^ as \x\ — >• -|-cxd to some dy- 
namical system having vr as its limiting point. In particular, for large |x|, 
X^ reaches any neighborhood of vr in a quasi-deterministic finite time. And 
this will show (Sections 6 and 7) that asymptotically, X is instantaneously 
at vr. 

The results of this section are quite standard, essentially based on law of 
large numbers principles. The simple underlying idea is that as far as X^ 
is only observed over a finite time window, since the number |x| of initial 
particles goes to infinity while the numbers of births and deaths within the 
given window remain of the order of 1 (time is not rescaled here) , the initial 
particles asymptotically dominate the system and mostly stay alive all along 
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the time window thus behaving as |x| independent Markov processes with 
generator Q. 

For the same reasons, the process X is not different, in the limit |x| — >• +00, 
from the process x = X/L of the spatial distribution of particles. The same 
convergence results hold for both processes; once proved for X , they easily 
extend to x- 

Formalizing the above argument, the following coupling is intuitively 
clear. It compares the general model to the "closed" one (with no births 
nor deaths, but only initial particles). As in Section 4, generator Q is in- 
dexed by its birth and death parameters A and /i. 

Lemma 5.1. For any x G N", there exists a coupling between the process 
X^ with initial state x and generator Qx^^, and the process U^ with initial 
state X and generator JIq.O; such that for t>0 and i = 1,. . . ,n, 

Uf{t) -N,{t) < X^t) < Uf{t) +Nx{t), 

where M\ and M^ are two Poisson processes with respective parameters A 
and /i. 

X'^ , moreover, satisfies 

\x\-M^it)<\X'={t)\<\x\+Mx{t). 

Proof. The case /x = is a straightforward consequence of the repre- 
sentation (4) of X from Section 4. Indeed if /x = 0, (4) gives for 1 <i <n, 

ur{t)<xfit)= Y. %,(.)=.}+ Yl hi.it-<^.)=^,^.<t}<ur{t)+^fx{t), 

l<k<\x\ fc>|x|+l 

where U^{t) is constructed as Ylk=i -"-{?*:(*)=«} ^^^ -^-^ ~ (cfc)^ ^ \x\ + 1). 

Moreover, one gets \U^{t)\ < \X^{t)\ < \U^(t)\ + J\fx{t) by summing up 
over i the previous first inequalities. The lemma is proved in this case since 
|C/^(i)| = \x\ for any t>0. 

The general case is then derived using the first part of Section 4. Indeed, 
consider X^ as the first component of a random process {X^,Y,Z) in N^" 
such that Y{0) = Z{0) = 0, X^ + y is some process with generator ^x,o and 
\Y + Z\ is some Poisson process A/"^ with intensity /x. The first part of the 
proof then applies to X^ + Y and gives, for t > 0, 

C/^(t) < A^(t) + Y{t) < U^t) +Afx{t) 

componentwise, as well as \x\ < \X^{t) + Y{t)\ < \x\ +Mx{t). 
The lemma follows by noticing that 

0<"t^i{t)<\Y{t)\<\Y{t) + Z{t)\=J\ff,{t), l<i<n. D 
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The two main results of this section concern the hitting time of some 
neighborhood of vr by the space renormahzed process X. Namely, for any 
positive (5, 

f5=inf{t>0:||X(t)-7r|| <5}. 
Recall that the analogous time with x iii place of X is denoted by Tg. 

Proposition 5.1. For any positive 6, there exists some deterministic 
time ts>0 such that 

lim F^{fs>ts) = 0. 

|x|— >-+oo 

The same result holds for the stopping time Tg . 

Proof. We refer to the proof of Proposition 5.2 below. Proposition 
5.1 is obtained in the same way, just changing Sn-, sn and t^ into 6, s = 
-l/r]log{5/2B) and ts = -l/r]log{6/iB). D 

The following more accurate result will be required for analyzing the 
subcritical case A < /i. 

Proposition 5.2. There exist two positive constants A andij such that, 
for any sequence of positive numbers {Sn,N> 1) satisfying 



lim 6]\i = and lim 6]\ivN = +oo, 

N-¥+oo N^+oo 



then 



lim 

7V-5-+00 



max Fx{Ts^>tN) 

-xeN":\x\=N 



where tjy = — log — - . 
r) A 



The same result holds for the stopping time T^^ . 

Proof. First consider a closed system, that is, assume A = /U = 0; the 
general case will then be deduced from Lemma 5.1. As in Lemma 5.1, let U^ 
be the closed process with initial state x G N" where |x| = N . In this case 
(4) becomes 

N 

C/nt)=J^lfe(t)=0, l<i<n,i>0, 

fc=i 

where ^fc, 1 < A; < A^, are independent Markov processes with the same gen- 
erator Q and different initial conditions: for any i, 1 < i < n, £,k{^) = i for Xj 
of the N indices A: = 1, . . . , A. 
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As introduced in Section 3, let {Pt, t>0) denote the transition semi- group 
associated to Q. The exponentially fast convergence of any irreducible finite 
state space Markov semi-group to its stationary distribution, tells existence 
of i? > and rj > such that 



max \Pt{j,i)-TTi\ <Be '^*, 

l<i,j<n 



f >0. 



In particular, maxi<jj<„|Psjv(j,^) -7^i\ < Sn/"^ for sn = -}j^og^. 

The outline of the proof for the closed case is the following: at time stv, all 
trajectories ^k, ^ < k < N, are very close to it in distribution (by the order 
of 6]\f). Since U^{t) represents the empirical distribution of the N particles 
at time t, the law of large numbers shows that for large A^, U^{sn) is also 
close to TT (by the same order), because 6n tends to not too fast 

Precisely, for any N >1 and x G N" such that \x\ = N, 



\\E{U^sn))-7t\ 



E 



N 



'IT 



1 ^ 
k=l 



F(6(S7V) 



'7T) 



< 



5n 



Thus, for any A^ > 1, using Chebyshev's inequality for the last step, 
F{\\U^sn) - vrll > Sn) < pf ||t/"(s7v) - E(^^(5^))|| > % 



n 

i=l 
n 



unsN)-mnsN))\>^-j- 



6%NyA ■ 



Independence of the processes {^kA ^ ^ < N) yields 
Var(C/f(s^)) 



N 

^Var(l|5^,^)=i|)< 
fc=i 



N 

T 



(bounding the variance of any Bernoulli random variable by 1/4). Finally, 

n 



(5) 



max F{\\U^{sN)-7T\\>dN)< 

x£N":\x\=N uj^ 



SlN- 



Now consider the process X^ associated to any family (Aj,//i, 1 < i < n) 
of parameters and any initial state x such that \x\ = N. Still denote by U^ 
the associated closed process with the same initial state x. 

Define ^a^ = ~n ^°S 45- '^^^ ^^^^ part of Lemma 5.1 implies that, for any 

N> 1, 



I^"(iiv)-vr||<||[/"(t7v)-vr|| + 



^{Mx{tN)+M^{tM)) 
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SO that 

IP^m^ > tjv) <p(||f/^-(iiv)-vr|| > ^) +p(||AAA(tjv)+AA^(tiv)|| > ^). 

By (5) the first term tends to zero uniformly in x as A^ goes to infinity, since 
tN is associated to S^f^ in the same way as sn was to 6n- The second one is 
also easily shown to converge to zero, using Chebyshev's inequality for the 
Poisson variable Mx{t]\i) +M^{tN), together with the relation d^y/N ':$> 1 
that implies N6n > ViV > 1/6n > ^tv- 

The first part of the proposition is thus proved with A = AB. 

Using the last assertion of Lemma 5.1, it is not difficult to show that the 
same result holds for Ts^^ . D 

We finally just mention for the sake of completeness (it will not be used 
in the sequel) the following result that describes the asymptotic dynamics, 
as 1x1—7- +00, of the empirical distribution of the particles; it evolves as the 
distribution, as function of time, of a Markov process with generator Q. 

Not surprisingly, this can be proved using the same standard arguments 
as for studying the M/M /oo queue under the Kelly scaling (see [12]). 

Proposition 5.3. Consider the processes {X^^{t),t > 0) associated with 
some sequence {xn,N >1) of initial states satisfying lim7v-H.+oo iv" ~ Z'' f^''" 
some p £V. 

For any T > 0, as N ^ +oo, (X^^(t),t > 0) converges in distribution 
with respect to the uniform norm topology on [0,T], to the deterministic 
trajectory 

p{t)=pPf 
In other words, for any positive 5, 



lim P sup WX^'^it)- pPt\\>5]=Q. 

The same convergence holds for the corresponding processes {x^^{t),t > 0), 
N> 1. 

6. The supercritical regime. This section deals with the supercritical 
regime X> p. As the next proposition shows, the instability of the system is 
straightforward in this case. Theorem 6.1 establishes an almost sure result 
describing the long-term behavior, and Theorem 6.2 presents a surprising 
phenomenon. 

Proposition 6.1. When X> p the process X is not ergodic. 
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Proof. Just remark, using Proposition 4.1, that ii x G N" is the initial 
state, 

L{t)>\x\+Mx{t)-M^{t). 

Hence for any initial state, L{t) almost surely goes to +00 as t tends to +00. 
D 

The following theorem gives an almost sure description of the divergence 
of X(t) for t large. Among other arguments, the proof makes use for the first 
time of the martingale estimate provided by Proposition 3.2, and involves 
the representations of X given in Section 4. 

Theorem 6.1. Assume A > /U. Then, for any initial state x £ N", the 
following convergence holds almost surely: 

hm ^ = (A-^)vr. 

Remark 6.1. This theorem has a double meaning: it tells almost sure 
convergence both of x(i) to vr and of L{t)/t to A — /u as t — )• +00. 



Proof of Theorem 6.1. Assume the theorem is true when ^u = 0. 
Then, using the notation of Theorem 4.1, t~^{Xf^Q{t) — X-Q{t)) converges 

a.s. to (A — /x)7r and the componentwise inequality Xf^ > Xf^ q — X- q derived 

from Theorem 4.1, implies that each Xf{t) tends to infinity almost surely 
as t goes to infinity. 

As a consequence, since |-^(t)| can increase only when some Xi{t) is zero, 
then, with probability 1, lim^^+oo 1-^(^)1 is finite and limt^^ac Z{t)/t = 0, 
so that 

^ X-{t) , Xfoit) - Xl,{t) 

hm = hm = = (A — fijir 

t— >+oo t t— >+oo t 

holds almost surely, which is the stated result. 

The theorem must now be proved in the case where ^ = 0. In this case 
with no deaths, using representation (4), the process X^ splits into two (in- 
dependent) processes: X^ = U^ + X-, where U^ is associated to a "closed" 
system with |x| particles moving independently, and X- has no initial par- 
ticles, birth rates A and null death rates. Then t~^U^{t) obviously tends to 
zero almost surely as t tends to infinity, and all that is left to show is that 
t~^X-{t) converges almost surely to Xn. 

So dropping for simplicity the superscript 0, consider the process X with 
initial state 0, birth rates A and null death rates. Equation (4) here becomes 
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for t > 0, 

X,{t) = Y^ l{^^(^t-a,)=i}, l<i<n, 

k=l 

where {S,k,k > 1) have initial distribution Y27=ii'^i/^)^i- 

It will first be shown that the analysis can be reduced to the case of 
stationary trajectories (i.e., the case when Aj/A = ttj for 1 < i < n) by using 
a coupling argument. 

Indeed, associate with each ^^ a stationary process .^^ with the same 
generator, such that ((CAnCfc))^ > 1) is a sequence of independent processes 
in {l,...,n}^, and, for k>l, ^k, C'k ^re coupled in the classical following 
way: ^k and ^^ are independent until the first time T^ when they meet, and 
after that stay equal for ever. Recall that the "coupling times" T^, A: > 1, 
are integrable. Moreover, assume the {^k,^k)^k > 1, independent from A/a. 

Define the process {X'{t),t > 0) on N" analogously to X, with the same 
A/a, but with ^^ in place of ^k (k > 1). Then, for each i £ {1, . . . ,n}, 



\x,it)-x'M 



J^xit) 



5Z ^^{^k{t-ok)=i} -Hi'k{t-cTu)=i}) 



k=l 



A/'aW 

- Yl ^{Tk>t-rTk}- 
k=l 



Denoting A(t) the last term, A{t) is exactly the number of customers at 
time t in an Af/G/oo queue with no customer at time 0, arrival process A/a, 
and services given by the i.i.d. integrable variables T^, k> 1. 

It is easily proved that A(t)/t converges almost surely to zero as t tends 
to infinity. It is then enough to prove a.s. convergence of process X' to Avr, 
and so we assume from now on that {S,k,k> 1) are stationary. 

Since L{t)/t =J\f\{t)/t converges a.s. to A as t tends to infinity, the prob- 
lem is equivalent to proving that x(i) converges almost surely to tt, that is, 
by Lemma 2.1 that 

Ve>0 ¥{3T<+oo:'it>T,H{t)<e) = l. 

This will be done using Borel-Cantelli lemma and showing that 

+ 00 

Ve>0 ^P(3tG[cjfc,(Tfc+i[:ii'(t)>e)<+oo. 
fc=i 

Writing, for any fixed e. 



(6) 



<P( H{ak) > I j +F(H{ak) < | and 3t £]ak,ak+i[: H{t) > e ], 
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we will show that both series associated with both terms in the right-hand 
side converge for e sufficiently small [which is enough by monotonicity of 
the left-hand side of (6)]. 

Let us begin with the first term. Note that for A; > 1, xi'^k) = ^i'^k)/k- 
Then due to Lemma 2.1, it is enough to show that, for small e and any 
ie{l,...,n}, 



(7) 



fc=i 



Xi{ak) 



k 



VTi 



>£ < +00. 



This is obtained by using Chernoff 's inequality, that we recall in Lemma 
6.1. 



Lemma 6.1 (Chernoff's inequality). Let Z^, 1 < h< k, be k independent 
random variables such that \Zh\ <l and E(Z/i) = for 1 <h< k. 

The following bound holds for any r] G [0, 2a] where o"^ = Var(^^^-^ Z^): 



h=l 



> 7?cr < 2e 



-,?2/4 



Write 

Xi{ak) 



1 ^r^ „{i) 
h=l 



^^ = rE^^ 



with 41 = l{«h(a,-a^)=i} -T^iA<i< n. 



Since ^h^^y 1, are stationary, then for each fixed i £ {1, . . . ,n} and k>l, 

(i) 

the k variables Zj^ j^, 1 <h<k, are i.i.d. centered random variables, bounded 
by 1 in modulus. (Notice that independence is only true in this stationary 
case.) 

We can thus apply Chernoff's inequality, which gives, for each fixed k and 



h 



Xi 



.(^k) 



TTi 



> e 



/ . ^k,h 



h=l 



>ke\ < 2e" 



2fc/(4l,,) 



if e < 2vi where f j = 7rj(l — ttj) is the common variance of the variables Zj^j^. 
Property (7) is then proved (for small e, hence for any e by monotonicity). 

Now it must be shown that the second term in the right-hand side of 
(6) is summable as well for e small enough. Here, the stationarity of the 
movements will play no special role. 

By definition of cr^, x{t) = X{t)/k for any t € [(Tfc,crfc+i[. Moreover, at is 
a stopping time for the Markov process {X{t),t > 0), because it is the first 
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time when L{t) = k. Hence the strong Markov property yields 
n(H{ak)<^and3te]ak,ak+i[:H{t)>e)< max P^.(r^<ai). 

-\ 2 J x&N": \x\=k and 

H{x/\x\,tt)<£/2 

Clearly, the last event only depends on cji and on the movements of the \x\ 
initial particles, so that by independence of these variable and processes, one 
obtains, for any x G N", 

where Tfj is the first time the entropy associated to the initial particles is 
larger than e. Then using Proposition 3.2 in the case of a closed system with 
6 = e/A, a = (A/6') A 1 and l = k gives 

for any x G N" such that |x| = k and H{x/\x\,tt) < e/2. The second term in 
(6) is thus summable over k for e small enough. D 

Along the preceding proof, we used ai, in the particular case ^u = 0, as an 
asymptotic lower bound (as the initial state grows to infinity) for the exit 
time of xit) from some neighborhood of vr. This is a very crude underesti- 
mation, as the following result shows that this exit time is actually infinite 
with high probability. 

Theorem 6.2. Assume X> fi, and fix 5 and e such that < 6 < e < Eq 
where £q is given by Lemma 2.2. Consider a sequence {xn,N > 1) with 
limAr_i._|_oo |2;Ar|/A^ = 1 and //(xAr/|xAr| , vr) < (5. Then 

hm P,^(r^ = +oo) = l. 

Proof. By definition of Tfj, fx^^iTfj < +oo)=Fa:^{3t >0:H{t) >e), 
and so we need to study the behavior of H{t) for all time t > 0. The idea of 
the proof is twofold: first, the estimate given by Proposition 3.2 is precise 
enough to show that Tfj is much larger than A^, say T^ > N'^. After this 
time, the initial particles are negligible, and Theorem 6.1 then gives a control 
on the rest of the trajectory by reducing the problem to the case where the 
system starts empty. So we use the following decomposition: 

^xATh < +oo) < ^xjTfj < a2) +P,^(3t > N^:H{t) > e). 

For the first term, Markov's inequality gives 

(8) F^jTf, < N') < eE^Je-^n/'''). 
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Let 6' < £ — 5. By choice of e and 5, and since H{xj\f/\xN\,'^) < 5, Propo- 
sition 3.2 shows that there exists a constant Cs' such that by choosing 
a = 1/{0N'^), for any A'" large enough and any l^, 

The choice of (.m requires some care; as N grows, it must be both of order 
|xAr| and smaUer than L[Tfj) with high probability. Since \xn\ ~ A^, write 
|xAr| = A'' + UN with UN = o{N), and choose (.n = N — ^JNvn with vn = 
\un\ V 1. With this choice, ^at ~ A^ and In — \xn\ -^ — oo. The first relation 
implies, since e — 5' — (^ > 0, 

lim (^&\^N\+'in\ogN-{e~5')llM ^ g. 
Af-s>+oo 

Moreover, since Tfj < To because e < Eq, Proposition 4.1 implies that L{Tfj) = 
LiO)+Mx{Tf,) -N^{Tfj), hence 

P,.^(L(r&) <£jv) = IPx^(|xjv| +Mx{Tfj) -M^{Tfj)<lN) 

< p(inf(A6.(t) -M^{t)) < £jv - Ixjvl), 

where the last bound vanishes because A > /x, and so inf(>o(A/'A(t) — J\f^{t)) 
is finite with probability one whereas (.n — \xn\ goes to — oo. It results that 
¥xj^{Tfj < N'^) goes to thanks to (8) and to the following inequality: 

and it has been shown that each term goes to 0. 

All that is left to prove now is that hmAr^+oo IPx-Ar(3t > A^^ : H{t) > e) = 0, 
or, by Lemma 2.1, that F^j^i^t > A^'^ : ||x(^) ~ ^i"!! ^ £) vanishes. After time 
A^^ , the initial particles are negligible since a number of new particles of the 
order of N"^ have arrived. So the behavior of the system will be similar to 
that of a system starting empty, to which we can apply Theorem 6.1 (since 
in this case the initial state is fixed). 

To formalize this argument, a coupling between the processes X^ and X^, 
for any x E N", is required. 

Lemma 6.2. For any x,y € N"' with x>y componentwise, it is possible 
to couple the two processes X^ and X^ in such a way that for any t > 0, 
L^{t) — Ly(t) < \x\ — \y\ and the inequality X^(i) > Xy{t) holds component- 
wise. 

The proof of this lemma is postponed at the end of the current proof. 
Let X^ be the process starting empty coupled with X^^, and let L^ = 
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|X0|, L^^ = |X^^|, x° = X^/L^ and x""^ = X^^/L^^ be the corresponding 
quantities. The triangular inequality gives 

P(3t>iV^||x''^(t)-7r|| >e) 
(9) < P(3t>iV2.||^.^.(^)_ ^0(^)11 >^/2) 

+ P(3t>iV2:||x°(t)-vr||>e/2). 

Theorem 6.1 states that ^^{t) converges to vr almost surely, which shows 
that the last term goes to 0. For the first term, write for each i = 1, . . . , n, 



where A^^(t) = L^^(t)-LO(t). Lemma 6.2 implies that |Xf^(i) -XO(t)| < 
A^^(t) < |xAr|, hence since the function z i— > z/{z + a) is decreasing for any 
a>0, 



\xrit)-xnt)\< ..,,... : ' < 



This yields in turn, using t > N"^ for the second inequality, 



< Pf 3t > A^2 . ^^ ^ > ^/2 ) 



<P inf (l + L7t)/t-A^7|x^|)<4/e 

\t>N^ 

Theorem 6.1 shows that L^{t)/t — t- A — /i almost surely as f — t- +oo, and 
]X'^ /\xn\ goes to infinity as A^ goes to infinity by choice of xn- Hence almost 
surely, 

lim inf (1 + L^(t)/t • iVVkivl) = +oo 

N^+oot>N^ 

and the theorem is proved. D 

We now fill in the gap in this proof by proving Lemma 6.2. 

Proof of Lemma 6.2. Process X admits the following representation 
as the solution of a system of integral equations: 

Xiit)=Xi{0)+Mx,{t)- f l{x,is-)>i}dNM 



j^i ■^0 fc=l j^i ■'^ k=l 



* ^s), 1< z< n, 
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where N\- and A/"^. , for i = 1, . . . ,n, are Poisson processes with respective 
parameters Aj and /Uj, and for (i,j) G{l,...,n}^, i^^j, {Mg..,k> 1) is a 
sequence of Poisson processes with parameter qij, ah these processes being 
independent. 

Now using the same Poisson processes for X^ and X^ , it is easy to check 
that the inequahties Xf{t) > Xf(t) true at t = are preserved at each jump 
of any of the Poisson processes involved, and that \X^\ — |X^| is decreasing 
over time. D 

The previous resuhs make it possible to establish the fluid regime of the 
system by studying the rescaled process Xjv defined by 

— . X X(Nt) 

(10) X^(t) = ^_^, t>0. 

In the following, L^ denotes the rescaled number of particles, that is, Li\f{t) = 
L{Nt)/N, and Xn = ^n/Ln are the corresponding proportions. Note that 
any fluid limit is discontinuous at 0"'" (so that strictly speaking, X does 
not have any fluid limit), because Proposition 5.1 will show that the fluid 
limit is at vr at time 0^, and Theorem 6.2 will imply that it stays forever 
proportional to vr. 

Corollary 6.1. Assume X> fi, and let x: [0, +oo[i-)'M" be defined by 

x(t) = (1 + (A - /x)t)7r. 

Then, for any sequence {x]\f,N > 1) with \xn\ = X, any s,t such that < 
s <t and any e > 0, 

lim P^^ ( sup \\Xn{u) - x{u)\\ >e)=0. 

Proof. Since the size of the initial state goes to infinity. Proposition 5.1 
shows that for any 5 > 0, the event {T^ < t^} occurs with high probability. 
Since T^ is a stopping time, the strong Markov property makes it possible 
to use Xt^ as a new initial point, which is as close to equilibrium as desired. 
Since, moreover, the total number of customers did not significantly evolve 
in this time interval, this initial point will satisfy the hypotheses of Theorem 
6.2, which makes it possible to conclude. 

Denote AAr(s,t) the distance of interest, 

(11) Aiv(s,t)= sup ||X^(u)-x(n)||. 

s<u<t 

First, the following decomposition makes it possible to consider all further 
convergences on the set {T5 < t^}: 

P,^(A^(s,t) >e)< F,^(A^(s,t) >e,Ti< t^) +P,^(r5 > t^) 
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and the last term goes to by Proposition 5.1. The strong Markov property 
used with the stopping time Tg then shows that 

¥^^{AN{s,t)>£,Ts<ts)<E.^^[¥xiTs){^N{0,t)>e)]. 
Now, we isolate the event of interest {\L{Ts) — \xj\f\\ < yN} by writing 

^xA^X(Ts)i^N{0,t) > e); \L{Ts) -\xn\\<Vn] 

< max Py(AAr(0,t)>e); 

yeN'-:\\y\-\xN\\<VN 

and i|j//|j/|-7r||<<5 

therefore, if we note y^v the value that realizes this maximum (the set over 
which the maximum is considered is finite). 



E,j¥x^Ts)iAN{0,t)>e)]<¥,^{\L{Ts)-\xN\\>VN)+FyJANiO,t)>e). 

The following inequality holds for any time n > and any initial state (Lem- 
ma 5.1): 

\Liu) - LiO)\<Mx{u) +M^{u) =Mx+^iu), 
and yields 

F,^{\L{Ts)-\xn\\>Vn) 

< P,^ {J^x+^,{Ts) >Vn,Ts< h) + P,^ {Ts > tg) 
<F{Afx+^,{ts) > Vn)+F^^{Ts > ts). 
This last sum vanishes, so that all is left to prove is that as A^ — )• +00, 

Fy^{AN{0,t)>£)=Fy^( sup \\XN{u)-x{u)\\>e)^0. 

^0<u<t ' 

Note that the initial state y^ is now such that \yfq\lN goes to 1 (because 

|xAr| = A^ and \\yN\ — \xn\\ < V^), and H{yj^ /\yj^!\^'iT) is as small as needed 

to apply Theorem 6.2, since ||yAr/|yAr| — 7r|| < 5 and 5 > is arbitrary small. 

The triangular inequality and the definition of x give for any < u < t, 

\\Xn{u) - X{u)\\ < \\Xn{u) - Ln{u)tt\\ + \\[Ln{u) - (1 + (A - fi)u)]TT\\ 

< \\Xn{u) -vr|| sup Ln{u) 

0<u<t 

+ ||7r|| sup |LAr(M) - (1 + (A -/i)u)|, 

0<u<t 

and so 

P,^(Ajv(0,t)>£) 
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(12) <IPyjv( sup ||xAr(^i)-vr|| >e/(2 sup Ln{u] 

^0<u<t ^ 0<u<t 

+ ¥y^( sup |Ljv(u)-(l + (A-/i)n)|>e/(2||7r| 

^0<n<t 

Under P^jy, a trivial upper bound for Ln{u) for < u < t is given by 

liv(n) < ^{\yN\ + Mx{Nt))= AN{t); 
therefore, for any constant C > 0, 

Vy^i sup \\xN{u)-T^\\>e/[2 sup Ln{u) 

^0<u<t ^ 0<u<t '' 

< Fy^ ( sup IIxtv W - vr|| > e/i2C)) + P(^7v(i) > C). 






0<u<t 



For any t > 0, A]\f{t) converges almost surely to 1 + At as A^ goes to infinity; 
therefore, ¥{Ai\i{t) > C) goes to for C = 2(1 + At). The other term vanishes 
as well. Indeed, 

Py. I sup llx^(n) - ^11 > e/i2C)) = F,^(r^/(2C) < ^^^^ 

and by Lemma 2.1, there exists some e' > such that T^'y'^'-^) > Tfj, hence 
P,^(T^/('^) < Nt) < Fy^iTi; < Nt) < Fy^iTf; < +oo). 

One can, moreover, assume e' < Eq without loss of generality. Observe that 
so far, 5 is arbitrary; it can be chosen small enough, say 5 <5o, so that using 
Lemma 2.1, H{yN/\yN\,'^) < e'/^j and yjy thus satisfies the hypotheses of 
Theorem 6.2, which shows that Fy^(Tff < +oo), and hence the first term in 
the upper bound of (12), vanishes in the limit N — )• +oo. 

The second term of (12) is easier to deal with. We reduce the problem to 
the event {To = +00} by using the following upper bound: 

sup \L{u)-il + iX- f,)u)\>e/i2\\7r\\)) 

d<u<t ' 

(13) <Pj,^(ro<+oo) 

+ Pj,^( sup |L(n)-(l + (A-/i)u)|>e/(2||7r||),ro = +oo 









0<n<t 



The first term Py^(7o < +00) in the right-hand side of (13) goes to since, 
by Lemma 2.2, Pj^^(7o < +00) < Fy^(^'^ < +00) which has just been proved 
to vanish as A^ — )• +00. 
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Because L{u) = L(0) +Mx{u) — A/'^(u) for all n > on {To = +00}, we get 
the following upper bound for the second term: 



P( sup 

vO<u<t 



T7(|y7v| +Mx{Nu) -M^{Nu)) - (1 + (A - /x)u) 



>^/(2|kl 



and this term goes to thanks to Doob's inequality. The proof is complete. 

D 

7. Stability of the subcritical regime. In this section we consider the 
subcritical regime A < //, that is, the case when the input rate is smaller than 
the maximal output rate. As in the previous section, the key ingredients are 
the short- and long-term "homogenization" property and the M/M/ 1-like 
behavior of the total number of customers. The next lemma will be useful 
for establishing the fluid behavior of the system. It gives a control on the 
stopping time Tf^, or equivalently T^; with high probability, T^ is larger 
than the time needed for a stable M/M /I queue to empty. 

Lemma 7.1. Assume A < /U. Fix some a > and let {x]\f,N > 1) be any 
sequence in N" such that 

\x Ml 

lim =a and lim H(xiv/\xm\,tt) =0. 

Then for any t < a/{fi — X) and any e < Eq where Eq is given by Lemma 2.2, 

hin P,^(r^<iVt)=0. 

Proof. Denote Hn = H{xn /\xn\^t^)^ and let {In,^ > 1) be a sequence 
of integers such that N ^ i^ ^ NH]y and ijy » log A'' [such a sequence 
clearly exists, for example, £j\f = Ny/H^V (logA)^]. Proposition 3.2 with 
a = l/N"^ and 5 = e/2 gives 

(14) E,.^(e-^^^/^';L(r&) > i^) < C^/^e^^^^g^+l^^l^^-^^^/^^ 

where the last bound goes to by choice of £n- Let now tn be defined by 
Tj\[ = inf{t > : L{t) < In}- Since £j\[ is an integer and L has jumps ±1, we 
have L{tn) = £n, and consequently, for any t > 0, 

E,Je-'^y^';L{m > £^) > E^Je-'^^y^'^Tf, < r^) 
(15) 

>e-^*/^P,^(r^<r;vAiVt). 

Inequalities (14) and (15) together imply that ¥x^{Tfj <tn A Nt) goes to 
as A^ goes to infinity. Since 

F^^iTfj < Nt) < F^^iTfj <tnA Nt) + F^^{tn < Nt), 
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all is left to prove is that Fx^^^tn < Nt) goes to if t < a/(/x — A). Using the 
lower bound L(t) > L(0) +Mx{t) — 7V)j(t) from Proposition 4.1 and the fact 
that |xAr| > {fi — X)Nt + ^7v for A^ large enough if t < a/(/i — A), we get for 
such a t, 

Fx^{TN<Nt) 

< P,^(3s G [0,Nt]:L{0) +Mx{s)-M^{s) < In) 

<P^^( sup (AA^(s)-AAA(s)-(/x-A)s)>|xjv|-(/U-A)iVt-£jv) 

^0<s<Nt ' 

<P,^( sup {M^{s)-Mx{s)-{ii-\)sf>{\xf,\-{iJi-\)m-i^f\. 

^0<s<Nt ' 

Since (A/'^(s) — N\{s) — (/i — A)s,s > 0) is a martingale, Doob's inequality 
yields that the last term is in turn upper bounded by 



Var AAA(iVt) + Var AA^(iVt) (A + ii)Nt 



0. 



(|x7v|-(/x-A)m-£jv)2 {a-{ix-\)tYN'^ □ 

The fluid behavior can now be established. Recall that the rescaled process 
Xat is defined by Xa? (t) = X{Nt)/N for any t > 0. In what follows, for u E M, 
u~^ denotes max(u, 0). 

Proposition 7.1. Let x: [0, +oo[i-)>M" be defined by 

x{t) = {l + {X-fi)t)^Tr. 
Then for all < s <t and all e > 0, 



lim 



AT^+oo LxeN" : |a;|=Af \s<u<t 



max Fx[ sup \\Xj^[u)—x{u)\\>e 



0. 



Proof. Lemma 7.1 makes it possible to study the system for t < l/(/i — 
A). An additional coupling argument, involving larger initial states, is then 
required to show that fluid limits stay at after that time. For this technical 
reason, initial states of size equivalent to aN for some a > will be consid- 
ered, and the following more general result will be established: for a > 0, let 
Xa : [0, +oo[i-)' M" be defined by 

Xait) = ia + {X-fx)t)^TT. 

It will be proved that for any a > 0, any s, t with < s < t and all e > 0, 



lim 



max P^( sup \\X]\[{u) — Xa{u)\\>e] 



LzeN":|z| = LaAfJ \s<u<t 

where the notation of the previous section are used. 



^0, 
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First assume t < ta = a/{fj. — A), and set Aiy{s,t) = sup3<„<J| XAr(u) — 
j;a(M)||. The first steps of tlie proof are similar to the underloaded regime, 
namely using the strong Markov property to replace the arbitrary initial 
state by some initial state with low entropy. More precisely, let 6n- and ^at 
be as in Proposition 5.2. For any x £ N" with \x\ = N, one has 

F^{AN{s,t)>e)<F^{AN{s,t)>e,Ts^<Ns)+F^{Ts^>Ns). 

Since tjy/N goes to 0, Proposition 5.2 gives that the last term Fx{Tsj^ > Ns) 
goes to uniformly in x G N" with \x\ = N. As for the first term, we write 

F^{AN{s,t)>e,Ts^<Ns) 

<E,.[Px(T,^)(Ajv(0,t)>e)] 

< Fy^iANiO,t) >e)+ F,{\L{Ts^) - L(0)| > Vn), 

where y^ S N* is such that 

Fy^{AN{0,t)>e)= max Pj,(Ajv(0,t) > e). 

j/eN": \\y\-laN\\<VN 
and ||j//|j/|-7r|j<5jv 

Because T^^ < tN with high probability, and because t^ /vN ^ 0, one can 
show similarly as in Section 6 that as A'^ goes to infinity. 



max F,{\L{Ts^)-Lm>^/N)^Q. 

xSN": \x\=N 

Along the same lines as in the overloaded case, one gets, by introducing the 
term Ln{u)tt, that for any C > 0, 



■ VN 

< 



(A7v(0,t)>e) 

( sup ||x7vM-^ll>£/(2C)) 



(16) 



S/ivl 

^0<u<t 



+ Fy^( sup Ln{u)>C 

^0<u<t 



+ ^yM 



( sup |L^(u)-(a + (A-/x)u)|>e/(2||7r||)). 



^0<u<t 



Note that since t < ta, Xa{u) = (a + (A — //)tt)7r for < n < t. For C large 
enough, Pj^^(supo<M<i-^Ar(u) > C) goes to as A" goes to infinity. Moreover, 
Lemma 2.1 gives 

^y^ ( sup ||x;v(^) - vrll > e/(2C)) = P,^(r^/(2C) < ^^) < ^^jr^e^ < ^^) 
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for some e' > that can be assumed to satisfy e' < Eq. Since the sequence 
(yiv, A^ > 1) satisfies the hypotheses of Lemma 7.1 and since t <ta, this last 
upper bound goes to 0. Moreover, since 



( sup \Ln{u) - (a + (A - fi)u)\ > e/(2|K||)) 
^o<«<t ^ 

<^y^{ro<Nt) 



VN 

^0<u<t 



+ . 



( sup |L7v(n)-(a + (A-;u)n)|>e/(2||7r||),ro>Aft 



VN , 

^0<-u<t 



we conclude, using Lemma 7.1 together with Lemma 2.2 for the first term, 
and Doob's inequality for the second one, that 

Fy^ ( sup \Ln{u) - (a + (A - ^i)u)\ > e/(2||7r||)) ^ 0. 

The proof in the case < s <t <ta is thus complete. 

To conclude in the other cases, a monotonicity argument derived from 
Lemma 6.2 is used. Let < s < t and t >ta, and assume in a first step 
that t — s < e/{2{fi — A)). In addition, let b > {fi — X)t be fixed, and let 
tb = b/{fi — A) be the corresponding time. Note that t >ta implies that 
6 > a, so that for any x G N" with |x| = [aN\ , there exists some y £ N" such 
that y>x componentwise and \y\ = \hN\. For such x,y, Lemma 6.2 shows 
that X^ and X^ can be coupled in such a way that |X^(t)| < |X^(t)| for 
any t > 0. Hence for any u> s, using the inequality ||f || < \v\ < n\\v\\ for any 
t; G M" , one gets 

\n{u) - Xa{u)\\ < |X^(n)| + \Xa{s)\ < \X%{u)\ + \Xa{s)\ 

< n\\X%{u) - Xbiu)\\ + \xi,{s)\ + \xa{s)\. 
By definition, 

\Xb{s)\ + \Xa{s)\ = (/X - X){tb -s) + {fl- X){ta - s)+ < 2{^l - X){tb - s) . 

This yields in turn 

Pj sup \\XN{u)-Xa{u)\\>e) <¥y( sup \\XN{u)-Xb{u)\\>e"^ 

where e" = (e — 2(/i — X)(tb — s))/n^ and finally, 

max P^( sup \\Xi^{u) — Xa{u)\\>e] 

■^■.\x\ = \aN\ \.<n,<i ) 



max P^ 



< max ¥yl sup \\X]\i{u) — Xb{u)\\> e"). 
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Since it has been assumed that t — s < e/(2(/x — A)), b > {fj. — X)t can be 
chosen small enough so that e" > 0. Since t <t{j, the first part of the proof 
implies that 



lim 



max 

.i^eN":|j/| = LbAfJ " \s<u<t 



y\ sup \\X]\[{u) -Xb{u)\\>s") 



0. 



This proves in particular that when < s < t and t — s < e/(2(// — A)), then 
max|^l=I^Q^j Pa;(A7v(s,i) > e) — )■ 0. It is now left to extend this result to any 
s,t such that s <t, which is a consequence of the following decomposition: 

Q 

max Fx(A]\f(s,t)> e) <y^( max Fx(An(sj-i,Sj) > e)] , 

xeN":\x\ = laN\ ^ ^ ' ^- ^ " ^ V^eN" : |a'| = La7VJ ^ ^^ ' ^^- V' 

where sq = s < si < • ■ • < Sq = t and Sj — Sj-i < e/(2(/i — A)) for 1 < j < q. 
Indeed, it has just been shown that each term of this finite sum goes to 0. 
D 

Remark 7.1. It can be proved that in the critical case A = /i > 0, the 
fluid limit is constant and equal to n, that is, if A = // > 0, then for all 
< s < t and ah e>0, 



lim 



max Fx( sup ||-'^Af(n) — 7r|| > e 

.I'SN" ; \x\=N \s<u<t 



0. 



This convergence follows readily from Proposition 7.1 and the following cou- 
pling. For < r/ < A, if X^ is a subcritical process with arrival rate X — r] 
and departure rate X = fi, then X and X^ can be coupled in such a way 
that ||X(t) — X {t)\\ < Nri{t) for alH > where N^j is a Poisson process with 
intensity rj. 

Note that the behavior of the fluid limit in the critical case does not make 
it possible to infer the stability or the transience of process X. The analogy 
with the M/M/1 queue nevertheless suggests that it could be recurrent null 
in this case. 

In contrast, the behavior of the fluid limit shows that X is ergodic in the 
subcritical case A < //. 

Proposition 7.2. When X< fi, the Markov process X is ergodic. 

Proof. According to Corollary 9.8, page 259 of [12], it is enough to 
show that for some deterministic time T > 0, 

lim max E^. (L^v (?")) = 0. 
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Recall that Ln(,T) = L{NT)/N, and let e > be fixed; then for x G N" with 

|x| =N, 

E, (Iat (T) ) < e + E, (I;v (T) ; Ijv (T) > e) 
<s + {l + XT)F^(LN{T)>e) 

+ ^]K,{Mx{NT) - XNT;Ln{T) > e), 

where the second inequality comes from L]y{T) < LpfiO) +J\f\{NT)/N. For 
any T > 0, using Cauchy-Schwarz inequality, an upper bound on the last 
term is given by 

1 1 /at" 

-E^{Mx{NT) - XNT;Ln{T) > e) < -E{\Afx{NT) - XNT\) <\Jj^, 



XT 




so that finally 

max E^(L^(T)) <£ + (! + AT) max F^(Ln{T) > e) -, ., ^ . 

and all that is left to prove is that for some T > 0, max|2,|=^Pa;(TAr(T) > e) 
goes to as iV grows to infinity; this is a direct consequence of Proposition 
7.1 with T = l/(/i — A) since x(T) = 0. The proof is now complete. D 



APPENDIX: MARTINGALE CONSTRUCTION 

This appendix is devoted to proving Theorem 3.1 which states the exis- 
tence of a fundamental family of local martingales. In Proposition A.l, we 
first establish the harmonicity of a special function g which has an integral 
form. Then a change of variables leads to the local martingale introduced in 
Theorem 3.1. 

A.l. An integral harmonic function. The starting point is the generator 
0, of the Markov process X given, for any x G N" and any function / : M" i— )• 

E, by 

n n 

+ ^ QijXiifix + Cj - Ci) - f{x)). 

In addition to the irreducibility of Q = {<lij)i<i,j<n-, we will require that 
Q is diagonalizable in C, that is, that there exists a set (a;j,l < j < n) 
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of eigenvectors of Q that generate M". The complex square matrix u = 
(wjj)i<jj<„ where ujj = (wj,j)i<j<n is invertible. 

We can assume without loss of generality that a;„ = 1, denoting by 1 the 
vector in R" with all coordinates equal to 1, so that a;„ is associated to the 
null eigenvalue; more generally, for 1 < j < ri, 9j will denote the (possibly 
complex) eigenvalue associated to (jJj. The negative trace of Q is then given 
by -e = YJl^i withe>0. 

In the sequel, % will denote the hyperplane of M" defined by 



n=\v(^W:^TTiVi = Q 



For j = 1,. . . ,n — l,ujj gT-L since Qojj = Ojujj for 6j ^ implies (in a matricial 
form where vr is a row and loj a column) that irujj = {6j)^^ttQujj which is 
since ttQ = 0. These n — 1 eigenvectors then generate 71. 

We recall some notation and results of Section 3. {Pt,t G M) denotes the Q- 
generated Markov semi-group of linear operators in W^:Pt = e**^, extended 
to all real indices t into a group. Each Pt has eigenvalues e •'^ and eigenvectors 
ijj, j = 1,. . . ,n. For any v £ M*^ and t > 0, we define 

cj){v, t) = {(Piiv, t),l<i<n) = P_tv. 

li V £ M" and ^{v,-) is any primitive of X]r=i[A*«*?^«(^' ')/(! + 4'i{vr)) — 
Xicpi^v, •)] on some open subset V of {t > 0:Vi = 1, . . . ,n, 1 + (j)i{v,t) ^ 0}, 
then the function hv(t,x) defined by 

n 

(17) K{t,x) = e'^^''^'^l[{l + Mv,t)r 

4 = 1 

is space-time harmonic with respect to Q in the domain V x N*" (see Propo- 
sition 3.1). 

The suitable domain of integration for constructing our martingale will 
be 

v{t) = {ve'H:i + 4>iv,t)>o}, teM, 

where, for any u £ M", n > (resp. n > 0) means that Uj > (resp. Ui > 0) 
for every i = l,. . . ,n. 

For each t G M, I'(t) is an open subset of 7i. Moreover, it is clear from the 
definition of T>{t) and from the invariance of Ti under the group of operators 
{Ps,s£ M) that for any v GW and any t > 0, 

vev{t) ^^ P^tveV{o). 

So for any t G M, V{t) = Pt(p{0)). Then since 2?(0) = {v G R'^-.Y^'^iriVi = 
0, 1 -h u > 0} is clearly bounded, each V{t) = -Pt(P(0)) for t G R is bounded 
as well. 
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Define the subset ^ of "^^ x R by 

A = {{v,t):tGR and w G P(t)}. 
Tlie first step is to sliow tliat the following choice of 93 makes sense: 

(18) ■i>{v,t)= 1" y(ni , ^''°:'' , -xMv.s)]ds 






+ (t)i{v,s) 



for (f , t) £ A. This is the object of the following two lemmas, which will also 
give some regularity properties of 93 in view of Proposition A.l. 

Lemma A.l. Ifiv,t) G M" x M satisfies l + (j){v,t) > and v ^ -1, then 
t + (j){v, s) > for all s <t. As a consequence, 



(19) t>s =^ V{t)cV{s), s,teM, 
and 

(20) v{to)=[jv{t), to em. 

t>to 

Proof. Let us first remark that the irreducibility of Q implies that 
for any r > and {i,j) £ {1, . . . ,n}, the probability Pr{i,j) that a Markov 
process with generator Q initiated at i is in state j at time r is positive. 
Indeed, if i = io,ii, . . . ,ik= j is a path from i to j such that qii_-^,ii > for 
I = 1,. . . ,k, then there is a positive probability that the process has exactly 
followed this path by time r. 

This implies that PrU > for any r > and u £ M" such that u > and 
u/0. 

Now let {v,t) satisfy the hypotheses in the lemma, then 1 + (p{v,t) / 
since 

0^1 + v = Pt{l + P^tv) = Pt{l + ^{v,t)) 

and the previous property applied to u = 1 + (p{v, t) and r = t — sforsKt 
gives 

1 + (I){V, S) = l + P-sV = Pt-s{t + P-tv) = Pt-s{t + Ct){v,t)) > 0. 



The implication t> s =^ T^{t) C 'D{s) follows, noticing that —1 ^ T-L. 

To show (20) for to G M, note that T)[tQ) contains the right-hand side 
union by (19), and that the reverse holds since, for v G Vito), the inequality 
1 + (f){v,tQ) > extends to some neighborhood of tg- D 
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Lemma A. 2. (i) For any i G {1, . . . ,n}, the two integrals 

(Pi[v,sjas ana / — r as 

are well defined for v € ^(0), continuous as functions of v on this domain 
and, respectively, bounded and bounded above on 'D(O). 
The function ipo can then be defined on T>{0) by 



/ ^("T 



Mv)= I y'^{fiirr^^^^^-kMv,s)]ds 



and is continuous and bounded above on 'D(O). 

(ii) The function ip given by (18) is well defined for {v,t) £ A and satisfies 

ip{v,t) = ^poiP-tv) {v,t)£A. 

If is bounded above on A and continuous with respect to v £ T){t) for fixed 

teR. 

Proof, (i) Notice that, for fixed v G M", the map s i— )■ 0(i; , s) = e^^^v is 
continuous on M (with values in M"). Moreover, if v gH, it has a fast decay 
as s tends to — oo as a consequence of the exponential fast convergence of 
Pt{i, •) to vr (already used in Section 5). 

There exist some positive constants ry and Ci such that, for any s < 0, 

(21) max \P-s{i,j) - vr J < Ci ■ e''". 

l<i,j<n 

This gives, for s < and v £71, 

(22) \\cPiv,s)\\<C2-e^^\vl 

where C2 = nCi which ensures the existence of the vectorial integral J_ (j){v, 
s) ds for any v gT-L. This integral is continuous with respect to u in ^ since, 
for V GTi, 

/ P-svds= I {P^s-'n)vds=( {P^s-'n)ds]v, 

J — 00 J —00 \J — oo / 

where II is the square matrix with all lines equal to vr = (vri, . . . ,'7r„), and 
the last matricial integral has a coefficientwise meaning [and is well defined 
due to (21)]. This shows the integral J_ (f)(v,s)ds as a linear function of 
V GTi, thus proving its continuity with respect to v gV.. The boundedness 
of this function on T>{0) follows since T>{0) has compact closure in Ti. 

For the second integral, Lemma A.l together with the condition v £T>(0) 
first ensure that 1 + (j){v, s) > for s <0. The existence of this integral then 
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again follows from the continuity of s i— t- (j){v, s) and from the exponential 
decay in (22). 

Let us now begin by proving that it is bounded above on 'D(O), writing 

_^l + (t)i{v,S) J_^l + (l)i{v,S) J_il + (l)i{v,S) 

and upperbounding each term. 

It is easy for the second one, since l + (j)i{v, s) > implies that (j)i{v, s)/(l + 
(piivjs)) ^ 1- I^ particular, 

_i l + (pi{v,s) 



The first term can be extended to w G ^(0) [again by Lemma A.l and by 
the exponential decay in (22)] and can be shown to be bounded on P(0). 
Indeed, for v S I'(O) and s < —1, 1 + 4'i(v, s) is positive and tends to 1 as s 
tends to — oo uniformly in t> G ^(0) since, by (22), 

(23) sup Mv,s)\\ <C2-e^" sup \\v\\. 



Then l + (j)i{v, s) is bounded below by some positive 5 for {v, s) £ T'(O) x ]— oo, 
—1] (by (23), this is the case on D(0)x ]— cxd, — k] for k large enough, and 



T>{0) X [— K,— 1] is compact), and the following bound holds for v £ ^(0), 
using (23): 



"'^^ ds 



l + (j)i{v,S 



^^2 II II r' r,s, C2e-^ II II 

< ^^ • sup ||n|| / e' as = — sup ||u| 



uGV{0) J~°° "^ mG^CO) 



Let us now show the continuity of J_^(j)i{v,s)/{1 + (l)i{v,s))ds with re- 
spect to t; G ^(0), using the continuity of (j)i{v,s) for fixed s, together with 
Lebesgue's dominated convergence theorem. The difficulty is that (/>j(f , s)/(l + 
(f)i{v,s)) is not clearly dominated uniformly in w G ^(0) by some integrable 
function of s on ]— oo,0], since for s close to and v close to the the portion 
of d'D{0) where 1 + Vi = 0, the ratio (pi{v,s)/{l + (pi{v,s)) goes to infinity 
and is not easily controlled. 

It is, however, possible to show local domination, using (20) in the par- 
ticular case to = and dominating the integrand on each 'D(t),t > 0. This 
will prove continuity on each T>{t), t > 0, hence continuity on P(0) since 
the 'D{t), t > 0, are open subsets of I'(O). The domination uses the same 
argument as in the last point; if t > 0, then 



6{t) = inf{l + <p,{v,s),{v,s)eV{t)x]-oo,0]} 
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is positive. Then, for any v G T^{t) and s < 0, 

(t)i{v,s) 



l + (l)i{v,s) 



^ ^2 „s II I 



where the right-hand side is integrable on ]— oo,0], and hence provides the 
required domination. 

(ii) We use the group structure of (P^, s G M) to rewrite both integrals as 

(j)i{v,s)ds= (j)i{v,s + t)ds= 4)i{P-tv , s) ds 

3 J —OO J —OO 



and 



/ 



l + (pi{v,s) J_^l + 4>i{v,s + t) J_^l + (pi{P.tV,s) 



which ensures their existence for Ptv G ^(0), that is, v G T^{t) or equivalently 
(f,t) G A, and proves the connexion between ip and (/^q, hence the stated 
properties of (/?. D 

The function (p is now legitimately defined by (18) for t G M and v G T^{t). 

For fixed i G M, 9?(-,t) is continuous (hence measurable) with respect to v 
in Vit) and bounded above on this domain. 

Reversely, for fixed v G ^(0), ip{v, ■) is clearly C^ on the interval ]— oo,t^,[, 
where t^, = sup{t > : 1 + (j){v, i) > 0} > [by continuity of (/)(f , •) on R] , and 
dip{v,t)/dt = ^"[/ij(/)j(7;,t)/(l + 4>i{v,t)) — Xi4)i{v,ty\ on this interval. 

The following proposition constitutes the first step in defining a new 
space-time harmonic function, obtained by integrating with respect to f G 
T){t) the parametrized family of functions h^ given by (17), with ip given by 
(18). 

As T){t) C % and % is an (n — l)-dimensional subspace of W^ which is 
isomorphic to M"~^ through the one-to-one linear mapping 

M"-i — > n 

U I > U= \u,-2_^ TTiUi/TTn I 

the new harmonic function will rather take the form of an integral over the 
following subset of M"~^: 

C{t) = H-\V{t)) = {u G M"-^ : u G V{t)} = {n G M""^ : 1 + (j){u, t) > 0}. 
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Proposition A.l. For any locally Lebesgue-integrable f on M."'^'^ , the 
function g{t,x) given by the formula 



g{t,x) = / hu{t,x)f{u)du 
JC{t) 

CW f=i 

is space-time harmonic in the domain [0,+oo[xN*". 

Proof, g is well defined on [0,+oo[xN*". Indeed UiC^ + M'^,t)T' is 
continuous on R""^ for fixed i > and x G N*", hence bounded on the 
bounded set C{t) [since C{t) corresponds to 2?(t) through H~^]. By Lemma 
A. 2, e^^''^' also is continuous and bounded on C{t) since (p{-,t) is continuous 
and bounded above on P(t). Then since / is locally integrable, the product 
of these three functions is integrable on C{t). 

We have to show that dg/dt exists and satisfies 

dgit,x)/dt + n{git,-)){x)=0. 
As a rough argument, one expects that 

(25) ^(i,^) = ^^ ^^t,x)-f{u)du. 

Indeed the additional derivation term resulting from the i-dependency of the 
domain C{t) is bound to vanish, since hu{t,x) is zero for u on the frontier of 
C{t) (recall that x G N*"). Therefore, since 17 commutes with integration. 



^{t,x) + n{git,-mx] 

at .ic{t) 



Bh- 

^{t,x) + n{h^{t,.)){x) 



■ fiu)du = 



by harmonicity of the functions h^, using Proposition 3.1 with V = [0,t„[. 

To make this rigorous, all that is needed is to prove (25), by fixing some 
arbitrary x S N*" and to ^ 0, and studying the ratio [g{to + 6,x) — g{to,x)]/S 
as 6 tends to zero. The monotonicity of the family of sets C{t) forces to dis- 
tinguish the two cases 5 > and (for to > 0) 5 <0. For the sake of shortness 
we will only present here the case 5 > 0, the other side being similar. Both 
cases make a repeated use of the mean-value theorem and of Lebesgue's 
dominated convergence theorem. 

To simplify notation, define for n e C(t), t > and x G N*", 

n 

/i(u, t, x) = /i^(t, x) = e^("'*) J](l + 0,(n, t))"\ 
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Note that h inherits the derivabiUty properties of ip with respect to t (the 
factor involving (p being C^ on M), for fixed u € C(0) and x G N*", h is C^ on 

]-00,tu[. 

Let us X G N*" and to ^ be fixed. For any positive 6, using the inclusion 
P(to + S) C T>{to) one can write, 

g{to + 6,x) -g{to,x) f h{u,to + 6,x) - h{u,to,x) 

- j{u)du 



^ -Icito+S) ^ 



I 



h{u,to,x) 

-j[u)du. 



C(to)\C(to+S) ^ 

Let us first show that the first term tends to J^,^ s^{u,to,x)f{u)du as 6 

tends to zero. Using the mean value theorem, since h{u, ■,x) is C^ on [0, to + 6] 
for u ^C{tQ + 6), this first term can be rewritten as 

dh 

-wriu, to + p{u)5, x)f{u) du 

Cito+5) Ot 

for some p{u) e]0, 1[ depending on u,to,x and 6. 

As 6 goes to zero, ^{u,to +p{u)5,x)f{u) tends to ^{u,to,x)f{u) and 
the indicator function of C{to + 5) tends to the indicator function of C{to) 
due equation (20) which obviously extends to the sets C{t). The convergence 
of the first term will then result from Lebesgue's theorem by computing [we 
omit the variable (u, t) under (j)] 



dh 
'dt 



(n,t,x) = e^("'*)X;(l + '^*r"'n(l + '^^-)"'(/^*-^*<^«(l + '^«) + ^*^)^ 



and then using the following domination: for < 6 <1 and u G M"^^ 
dh 



dt 



{u,to+p{u)6,x)f{u)lc(^to+5) 



< 



k{to,x)-\f{u)\ ■tc(to), 



where the right-hand side is integrable on M" , and k{to,x) holds for 



sup e^ X sup 

A D{to)x[o,to+i] 



j2il+c|^^r^'ll(^+cP,^(^^^i-^Mi+cP^)+x,^'^ 



j=l j=/=i 



The convergence of the first term is thus proved. 

We now prove that the second term vanishes as 5 tends to 0. For any 
u G C{to) \ C(to + S) there exists some index i (depending on u) such that 
1 + 4>i{u, to) > while 1 + <pi{u, to + S) < 0, and this implies by the mean value 
theorem that <1 + (j)i{u,to) < —S-^{u,to + q{u)6) for some g(u)G]0,l[ 
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depending on u,to and 5. One can deduce the following upper bound, again 
assuming < 6 <1: 

I' h{u,t o,x) 

'c(to)\C{to+5) S 



-f{u)du 



<kl \f{u)\du, 

X{to)\C(to+S) 



where k is the following constant: 



supe''' X max 



A 



K'Kn 



sup 

O{to)x[0,to+1] 



dt 



X sup 

X)(to)x{t„} 



(1 






The right-hand side of the previous inequality converges to zero, again by 
Lebesgue's theorem, because / is integrable on the bounded set C(to)) and 
the sets C(to) \C(to + ^) decrease to as 5 decreases to zero, due to relation 
(20). D 

A. 2. Change of variables. The last step is now a change of variable in 
the harmonic function given by the integral (24), for a suitable choice of / 
so as to separate the time and space variables. 

It informally consists in choosing as new variables the quantities 7rj(l + 
4>i{v, t)) (1 < i < re), changing the domain T>{t) into V = {v ^ M" : f > and 
^"t;, = 1}. Formally, it will be slightly more complicated due to integration 
with respect to Lebesgue's measure on subdomains of W^~^ [the C(t)'s], 
which forces a round trip from M"^-*^ through M". So to be correct, this 
change of variable will rather transform the domain C{t) into the following 
one: 

(26) 5= JnGM"-^:n> and ^Ui<l I. 

We need to introduce some additional notation. Denote by A the diagonal 
re X re square matrix having 7ri,...,7r„ as its diagonal elements, by J the 
projection 

M*^ — > W-^ 
■ (t;i,...,t;„)i — > {vi, . . . ,Vn-i) 

and by /C, the hyperplane of M" defined by 



/C: 



v€. 



1=1 



1 



/C corresponds to M" ^ through the one-to-one affine transformation (anal- 
ogous to H from M"~^ toll), 

n-l \ 



K: 



u ■■ 



u, 



Ui 
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Notice that the inverse mapping of K (resp. H) is given by the restriction 
of J to /C (resp. Ti). The announced change of variable is given by the 
t-depending transformation 



pn— 1 



pn— 1 



^t: 



~ u I — > JA(P_iiJu + l). 
The next lemma shows that ^t can be considered for a change of variables. 



Lemma A. 3. For any t'>{^,^tisa one-to-one affine transformation on 
M""^"*^ which inverse mapping is given by 

^^^{u) = JPt{A-'^Ku - 1) 
and which Jacobian is Jac(^i) = 6^*11^=^1 ^«- Moreover, ^i(C(t)) =S. 

Proof. Since "^t is clearly an affine transformation in M""^, its Jacobian 
is the one of its linear part JAP^fH. Now J A = A', J where A' is the diagonal 
(n — 1) X (n — 1) square matrix having vri, . . . , 7r„_i as its diagonal elements, 
so that 



''n— 1 



/n— 1 



jac( jAP_iif ) = n ^0 Jac( jp-ti^) = n ^0 Jac(^-* 



jt 



■,i=l 



a=l 



n-1 

n 



TTi. 



The second equality results from the facts that J restricted to Ti coin- 
cides with H~^ and that Ti is generated by the first n — 1 eigenvectors of 
P-t, so that Ti is invariant under P-t, which restriction to Ti has Jaco- 



bian nr=i e 



n 



i=i' 



Jac(P_ 



e ) smce 



6n = 0. In particular 



Jac(^t) 7^ so that ^i is invertible. 

The formula for ^~^ easily results from the fact that A(- -|- 1) maps Ti 
onto /C and that the inverse mapping of K is given by the restriction of J 
to/C. 

Now using C{t) = {u £ M""^ : 1 + P^tHu > 0} together with the facts that 
A preserves the relation v > and (again) that A(- -|- 1) maps T-i onto /C, 
one gets that ^((C(t)) is included in J{{v £ IC:v > 0}) = S. Equality results 
from a similar argument for ^^ (5) C C{t). D 

The transformation ^t hence corresponds to the two following diagrams: 



A(P_r+l) 



nc 



Pt{A- 



-1) 



/Cc 



H 



C(t)c 



on— 1 



--^Sc. 



K 



un— 1 



C{t)cW-'^--Sc 



nn— 1 
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All that is left now is to choose for (24) a family of locally integrable 
functions in M""^ which behave nicely with respect to the change of variable 
^t. It will be given by the functions f°'~^ for positive a's, where, for u G M"~^ 
and V £ M", 



f{u) = ^{Hu) = ij{u) and V('^) = JJ ' (^"^ 



n— 1 

1 



Here, for any v G M" and 1 < i < ra, f, denotes the ith coordinate of v, so 
that the {co~^v)i^s (1 < i < n — 1) are the first n — 1 coordinates of v in 
the base (wi, . . . ,a;„) of eigenvectors of Q. As will become clear in (29), the 
next lemma establishes the property of '0 that makes it behave nicely with 
respect to the change of variables given by ^t by isolating the dependency 
in time in a separate factor: 

Lemma A. 4. For any t > and v G M", ip{Ptv) = e~^*V(^) • 

Proof. This result stems from diagonalizing Pt as uj~^PtUJ = e~ where 
Q is the diagonal n x n square matrix having 9i, . . . ,9n as its diagonal ele- 
ments. This readily gives ip{Ptv) = Y[7=i \i^~^^^~^v)i\ = e~^^'tp{v). D 

The main technical point is to establish that f"~^ is locally integrable; 
the next lemma provides in addition a useful upper bound. 

Lemma A. 5. /"~^ is locally integrable on W"'^^ for any a > 0. Moreover, 
for any compact set T C M"~^, 

(27) sup (a" //(u)"-^(iu) <+oo. 

0<a<lV Jt J 

Proof. If a > 1, /"~^ is continuous on M""^, hence locally integrable. 
So consider only the case when < a < 1. 

If the matrix cj has real coefficients (it can be chosen as such when the 
eigenvalues 6j of Q are real, which is, in particular, the case for a reversible 
Q)i /"~^ is easily shown to be integrable on any compact set T of M"""*^ by 
operating the change of variable 

which is linear and one-to-one, and transforms Jj, f {u)'^~^ du into the in- 
tegral over some compact subset of R"~^ of the locally integrable function 
nr=i l"'^*!""'^ (^P to the Jacobian constant factor). Then by considering A 
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large enough so that Ju)'^H(T) C [-^,yl]"~^ and ^ > 1, (27) is obtained 
from the fact that 

/n— 1 

This is not directly possible when uj has nonreal coefficients. In this case 
we can show that /"^^ is upper bounded by YVi=i l(-^^)*l"~"^ ^^^ some in- 
vertible (n — 1) x (n — 1) square matrix L with real coefficients. The change 
of variable u i— )■ Lu is then possible, showing (27) in this case similarly as 
before, which implies the local integrability of /"~^ for < a < 1. 
Since a < 1, this amounts to lower bounding / by YVtZi K-^"")*!- 
Call C the complex invertible (n — 1) x (n — 1) square matrix associated 
to the linear mapping Juj^^H on M"'^-'^, so that f{u) = YYjZi\iCu)j\, and 
write C = A + iB where A and B are real square matrices. For any p£ [0,1], 
u G M""^ and j € {1, . . . , n — 1}, the following inequalities hold: 

I {Cu)j I > max{ I {Au)j \,\{Bu)j\}> \p{Au)j + {I - p) {Bu)j \ 

= \{{pA + (l-p)B)u)j\. 

All that is left now is to prove the existence of some p G [0, 1] such that 
pA + (1 — p)B is invertible. It is done through considering the degree n — 1 
polynomial with complex variable, det(A + zB), which is nonzero at z = i 
(since detC ^ 0), hence not equal to the null polynomial. It then cannot be 
zero on the whole real interval [0, 1], which gives the result. D 

Proof of Theorem 3.1. The previous lemma shows that /"~^ is a 
suitable function to plug in (24); since (p{v,t) = (/9o(-P-tw) for {v,t) £ A, 
rewriting (24) and using the definition of / gives 

„ n 

g{t,x)= / e'^("'*)r[(l + </'^(^'Or/«"'^^ 

„ n 

= / e^«(^-'^")lT(l + P_t/7u)fV(-Pt^-i^w)"-idw. 
Jc{t) fJi 

Expressing P^tHu through "^tu for u G M"~^, one gets, since K is the inverse 
of J restricted to /C and A{P^tHu + l)efC, 

(28) P.tHu = A-^K^tu-l, u£C{t), 

so that operating the change of variables given by ^t yields by Lemma A. 3 

n— 1 „ n 

(29) g{t,x) = e''r\7r, / e^'>^^'"''-^^r\{A-'u)^^^{Pt{A-'u - l)r-' du. 
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Since a;„ = 1, we have ^(u — 1) = tl^iv) for any v G M", hence tlj{Pt{A~^u — 
1)) = ip{PtA~^u) = e~^^ip{A~^u), where the last equahty conies from Lem- 
ma A. 4. The following function g' , which only differs from 5 by a multiplica- 
tive factor, is thus again space-time harmonic: 

„ n 

= e-"''* / G{u) n ( - ) H^~'ur~' du, 
where we have defined 

Hence defining F as 

F{v) = ^j{A-^v), vgW, 

yields exactly the local martingale of Theorem 3.1. The second expression 
(2) is easily obtained. All one needs to do to complete the proof of Theorem 
3.1 is to check the announced properties of these two functions F and G. 

First, G is continuous and bounded on V, since, by Lemma A. 2, ipo is 
continuous and bounded above on T>{0) [if v gV, then A^^v — 1 € "H and 
A"^^ > 0, so that A-'^v -l£ P(0)]. 

Moreover, F is clearly positive and continuous on M", and thus bounded 
on the bounded subset V of M", and so (3) is the only property left to be 
checked. 

Relation (28) and the fact that ■0(- — 1) = V'(")) together with the defi- 
nitions of F and /, yield that F{u) = /{"^q u) according to the following 
steps: 

F{u) = iP{A-^Ku) = i^iA'^K'^oi'^^^u) - 1) = Tp{H%^u) = f{%^u). 

It follows by the change of variables induced by ^0 that, for a < 1, 

Js Js Jt 



where T = ^I'q (5). (3) then follows using Lemma A. 5. D 
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